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(57) Abstract 

An asynchronous transfer mode inverse multiplexed communication system is disclosed wherein a series of communication cells are 
multiplexed over a set of communication links. Each communication cell induces a framing bit of a predetermined framing bit stream for 
each communication link and a control channel bit of a control message for each communication link. Inbound communication cells from 
each communication link are aligned according to the corresponding framing bit stream. The control message specifies an ordered list of 
logical identifiers to indicate a multiplexed sequence of transfer of the communication cells over the communication links. 
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ATM communication with Inverse multiplexing over multiple links 
BACKGROUND Of THE INVENTION 

1. field Qf The Invention; 

The present invention pertains to the field of digital 
communications. More particularly, this invention relates to a system 
for inverse multiplexing data transmission over multiple 
communication links in an asynchronous transfer mode 
communication network. 

2. BACKGROUND: 

Packet switching communication networks are commonly 
employed to transfer digital information over long distances. Such 
packet switching communication networks are also referred to as cell 
switching communication networks. One type of packet switching 
communication network is an asynchronous transfer mode 
communication network. An asynchronous transfer mode 
communication network enables the transfer of digital voice 
information, digital video information and digital data over short or 
long distances via common carrier communication links. Such a 
communication network enables a wide variety of communication 
devices to share common carrier communication links on a demand 
driven basis. For example, such asynchronous transfer mode 
communication networks are commonly employed for a wide area 
network (WAN) communication. 

The common carrier communication links employed in such 
asynchronous transfer mode communication networks typically 
include relatively low speed metal wire based communication links. 
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One such type of relatively low speed common carrier 
communication link available in North America is the Tl 
communication link. A typical Tl communication link provides a 
maximum data rate of 1.544 megabits per second. Similarly, the El 
common carrier communication links available in Europe provide a 
relatively low maximum data rate of 2.048 megabits per second. 

In addition, the common carrier communication links 
employed in such asynchronous transfer mode communication 
networks usually include higher speed optical based communication 
links. An example of such a high speed common carrier 
communication link available in North America is the T3 
communication link which provides a maximum data rate of 45 
megabits per second. An example of a high speed common carrier 
communication link available in Europe is the E3 communication 
link which provides a maximum data rate of 34 megabits per second. 

Prior asynchronous transfer mode communication networks 
that require high bandwidth communication links typically employ 
T3 or E3 communication links rather than the lower speed Tl or El 
communication links. Unfortunately, the subscription rates for such 
T3 common carrier communication links are usually much higher 
than the subscription rates for Tl or El links. Moreover, in North 
America such T3 common carrier communication links are not as 
widely available as Tl common carrier communication links. Worse 
yet, in Europe E3 common carrier communication links are rarely 
available and El common carrier communication links have only 
recently become widely available. 

As a consequence, communication services requiring such high 
bandwidth communication links typically require a high premium 
payment for such high speed common carrier communication links. 
Moreover, such required high speed communication links may not be 
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available in some areas for such an asynchronous transfer mode 
communication network. 

In addition, some prior asynchronous transfer mode 
communication networks require communication links with a 
bandwidth higher than is provided by low speed common carrier 
communication links such as Tl or El but do not require the high 
bandwidth of the T3 or E3 common carrier communication links. 
Unfortunately, communication services requiring such medium level 
bandwidth communication links must typically employ expensive 
high speed common carrier communication links even though the 
full bandwidth range is not needed. For example, an asynchronous 
transfer mode communication network requiring a maximum data 
rate of 10 or 20 megabits per second must usually employ high speed 
T3 or E3 common carrier communication links. Such high speed 
common carrier links unnecessarily increase the cost of 
communication for subscribers to such an asynchronous transfer 
mode communication networks. 
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Summary and Obtects of th h invention 

One object of the present invention is to carry high speed 
asynchronous transfer mode connections over multiple low speed 
communication links. 

Another object of the present invention is to enable 
asynchronous transfer mode communication over multiple low speed 
communication links and to compensate for the differing delays 
associated with the multiple communication links. 

A further object of the present invention is to dynamically 
reconfigure the number of active links of the multiple low speed 
communication links as errors are detected on low speed 
communication links. 

A further object of the present invention is to remove 
individual low speed communication links that fail. 

Another object of the present invention is to restore a 
previously failed low speed communication link after removal of the 
failure by, for example, a common carrier reroute. 

Another object of the present invention is to dynamically 
reconfigure the multiple low speed communication links according to 
the bandwidth capacity required between peak capacity periods in the 
asynchronous transfer mode communication network. 

Another object of the present invention is to properly 
reconstruct a cell transmission sequence in an asynchronous transfer 
mode communication network while compensating for differing 
delays associated with each individual low speed comim ication 
link. 
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These and other objects of the invention are provided by an 
asynchronous transfer mode inverse multiplexed communication 
system, wherein a series of communication cells are multiplexed over 
each of a set of communication links. Each communication cell 
includes a framing bit such that the framing bits provide a 
predetermined framing bit stream for each communication link. On 
the receiving side, the inbound communication cells from each 
communication link are aligned according to the corresponding 
framing bit stream. The alignment provided by the framing bit 
streams enables compensation for the differing delays associated with 
each communication link. 

Each communication cell includes a control channel bit such 
that the control channel bits provide a control message for each 
communication link. The control message specifies an ordered list of 
the logical identifiers such that the ordered list indicates a multiplexed 
sequence of transfer of the communication cells over the 
communication links. Control messages are used to delete a failed 
communication link from the set of communication links and to 
restore the failed communication link after removal of the failure. 

The predetermined framing bit stream enables the 
determination of differential delays among multiple communication 
links. If the differential delays are excessive and beyond a maximum 
allowable limit, then the inverse multiplexed connection may be 
rejected. If the differential delays are within a permissible limit, then 
the framing bit stream is used to compensate for the relative delays to 
recreate the original high speed cell sequence. The framing bit stream 
also enables the alignment of control channel messages. 

Other objects, features and advantages of the present invention 
will be apparent from the accompanying drawings, and from the 
detailed description that follows below. 
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Brief Description of t he Drawtn^s 

The present invention is illustrated by way of example and not 
limitation in the figures of the accompanying drawings in which like 
references indicate similar elements, and in which: 

Figure 1 illustrates a communication network for one 
embodiment which comprises a set of communication nodes coupled 
for communication over a set of communication links; 

Figure 2 illustrates the outbound communication portions of a 
communication node for one embodiment including an outbound 
cell buffer, a state machine, and a set of framing and line interface (I/F) 
circuits; 

Figure 3 illustrates the inbound communication portions of a 
communication node including a set of framing and line interface 
circuits, a state machine, and an inbound delay compensation; 

Figure 4 illustrates the format of each communication cell 
transferred over the communication links wherein each 
communication cell comprises 53 bytes including a 5 byte header and a 
48 byte payload; 

Figure 5 illustrates the master framing bit stream carried over 
each of the communication links 1-N for one embodiment wherein 
the master framing bit stream comprises a bundle framing bit from 
each of a series of 128 communication cells (shown as cell 0 - cell 127) 
carried over each of the communication links; 

Figure 6 illustrates the control message transferred over each of 
the communication links wherein the control messages are carried 
via two master frames of inbound communication cells over each of 
the communication links; 



NSDOCID: <WO 96081 20A1J_> 



WO 96/08120 



PCT/US9S/10218 



-7- 

Figure 7 illustrates the format of a master framing bit stream 
which is used to determine the relative transmission delays among 
the communication links and to realign the cell streams transmitted 
over the communication links into the inbound delay compensation 
buffer; 

Figure 8 illustrates the control channel message which carries 
fixed length control messages between the master communication 
node and the slave communication node; 

Figure 9 illustrates the complete master framing sequence 
provided by the bundle framing bits and a complete control message 
provided by the channel control bits; 

Figure 10 illustrates the architecture of the inbound delay 
compensation buffer for one embodiment which comprises a set of 
link buffers that correspond to the communication links; 

Figure 11a illustrates a sequence of outbound communication 
cell numbers that comprise cell numbers 11, 10, 9, 8, 7, 6, and 5 from 
the outbound cell buffer; 

Figure lib illustrates the format of a control message carried 
over each of the communication links corresponding to the link 
identifiers 7,8,10 and 11. 
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Petailed Description 

Figure 1 illustrates a communication network 20 for one 
embodiment. The communication network 20 comprises a set of 
communication nodes 22-32 coupled for communication over a set of 
communication links 40-48 and a set of communication links 50. The 
communication network 20 enables communication according to an 
asynchronous transfer mode communication protocol. 

The communication nodes 26 and 28 enable point to point 
inversed multiplexed asynchronous transfer mode communication 
via the communication links 50. The communication links 50 
comprise a set of communication links 1-N. For one embodiment, 
each of the communication links 1-N comprises a Tl common carrier 
communication link. For another embodiment, each of the 
communication links 1-N comprises an El common carrier 
communication link. 

The communication nodes 26 and 28 exchange streams of 
communication cells via the communication links 1-N. The 
aggregate bandwidth of the communication links 1-N provides point 
to point communications between the communication nodes 26 and 
28 at an effective data rate of N times the data rate of each of the 
individual communication links 1-N. For example, in an 
embodiment wherein the communication links 50 comprise a set of 
four Tl common carrier communication links the maximum 
effective data rate communication between the communication nodes 
26 and 28 is 4 x 1.536 megabits per second. 

The communication node 26 transfers outbound sequences of 
communication cells to the communication node 28 by inverse 
multiplexing the outbound communication cells over the 
communication links 1-N in a round robin fashion. The 
communication node 28 receives inversed multiplexed inbound 
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communication cells from the communication node 26 over the 
communication links 1-N and reconstructs the original sequence of 
communication cells. The communication node 28 compensates for 
the differing delays associated with each of the communication links 
1-N. 

Similarly, the communication node 28 inverses multiplexes 
outbound sequences of communication cells to the -communication 
node 26 over the communication links 1-N in a round robin fashion. 
The communication node 28 receives the inversed multiplexed 
inbound communication cells over the communication links 1-N and 
reconstructs the original sequence of communication cells while 
compensating for the differing delays associated with each of the 
communication links 1-N. 

The communication nodes 26 and 28 transmit master framing 
sequences over each of the communication links 1-N to coordinate 
the transfer of communication cells over the communication links 1- 
N. For one embodiment, the master framing sequences each comprise 
1 bit from each communication cells transferred over each 
communication link 1-N. 

The communication nodes 26 and 28 exchange channel control 
information via channel control messages transferred over the 
communication links 1-N. For one embodiment, the channel control 
messages comprise one bit from each communication cell transferred 
over each communication link 1-N. The channel control messaging 
structure designates one of the communication nodes 26 or 28 as the 
"master" communication node and designates the other of the 
communication nodes 26 or 28 as the "slave" communication node. 

The communication nodes 26 and 28 provide full duplex 
inversed multiplexed asynchronous transfer mode communication 
via the communication links 1-N. For purposes of illustration, 
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however, only communication cell transmission from the 
communication node 26 to the communication node 28 is hereinafter 
described. The communication node 26 is hereinafter designated the 
master communication node and the communication node 28 is 
hereinafter designated the slave communication node. 

Figure 2 illustrates outbound communication portions of the 
communication node 26 for one embodiment. The portions of the 
communication node 26 involved in outbound communication cell 
transfer over the communication links 1-N comprise an outbound 
cell buffer 60, a state machine 62, and a set of framing and line 
interface (I/F) circuits 70-72. 

The outbound cell buffer 60 buffers outbound communication 
cells for transfer to the communication node 28. The state machine 62 
sequences the outbound communication cells from the outbound cell 
buffer 60 to the framing and line interface circuits 70-72 in a round 
robin fashion that defines the inverse multiplexed outbound 
communication cell sequence. 

The state machine 62 inserts idle communication cells into the 
outbound communication cell streams on the communication links 
1-N if outbound communication cells are not available from the 
outbound cell buffer 60. The idle cells maintain the data rate available 
on each communication link 1-N. 

The state machine 62 also performs a master framing function 
for the inverse multiplexed outbound communication cells by 
manipulating individual bits of each outbound communication cell. 
In addition, the state machine 62 manipulates individual bits in each 
outbound communication cell to perform channel control messaging 
to the communication node 28. 
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For one embodiment, the framing and line interface circuits 70- 
72 perform Tl framing functions and line interface functions for 
transmitting outbound communication cells over the communication 
links 1-N. 

Figure 3 illustrates inbound communication portions of the 
communication node 28 for one embodiment. The inbound 
communication portions of the communication node 28 comprise a 
set of framing and line interface circuits 80-82, a state machine 84, an 
inbound delay compensation buffer 96, and an inbound cell buffer 86. 

The framing and line interface circuits 80-82 receive 
communication cells over the communication links 1-N and perform 
line interface and Tl framing functions for the communication links 
1-N. 

The state machine 84 sequences the inbound communication 
cells from the framing and line interface circuits 80-82 into the 
inbound delay compensation buffer 96. The state machine 84 filters 
out idle communication cells received over the communication links 
1-N. The state machine 84 extracts the inbound master framing 
sequence on each communication link 1-N and stores the inbound 
communication cells into inbound delay compensation buffer 96 
according to the corresponding master framing sequence. The state 
machine 84 also extracts the channel control messages for the inbound 
communication cells transferred via the communication links 1-N. 

The inbound delay compensation buffer 96 accommodates full 
master framing sequences of inbound communication cells and 
thereby permits delay compensation among the communication links 
1-N. Accordingly, the size of the delay compensation buffer 96 varies 
according to the length of master framing sequence required for the 
maximum permissible differential delay. 
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Figure 4 illustrates the format of each communication cell 
transferred over the communication links 1-N. Each communication 
cell comprises 53 bytes including a 5 byte header and a 48 byte payload. 
The 5 byte header includes an 8 bit cyclic redundancy check (CRC 8). 

The header of each communication cell includes a bundle 
framing (BF) bit and a channel control (CC) bit- The bundle framing 
bits of the communication cell streams transferred over the 
communication links 1-N provides a master framing sequence for 
each of the communication links 1-N. The channel control bits of the 
communication cell streams transferred over the communication 
links 1-N provides control messages between the communication 
nodes 26 and 28 via each of the communication links 1-N. 

For one embodiment, the bundle framing and channel control 
bits are located within the generic flow control (GFC) nibble of each 
communication cell header. For other embodiments, the bundle 
framing bit and the control channel bit may be located within the 
virtual path identifier (VPI) or virtual circuit identifier (VCI) portions 
of the communication cell headers. 

Figure S illustrates the master framing bit stream carried over 
each of the communication links 1-N for one embodiment. The 
master framing bit stream comprises a bundle framing bit from each 
of a series of 128 communication cells (shown as cell 0 - cell 127) 
carried over each of the communication links 1-N. A series of 128 
communication cells aligned to the master framing bit stream is 
referred to as a master frame. The receiving communication node 
extracts the inbound bundle framing bits BF[0] - BF[127] to identify 
master frame boundaries. 

The master framing bit stream is used to align the inbound 
communication cells into the delay compensation buffer 96. The bit 
phase alignment between the bits of the master framing bit stream and 
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the bundle framing bits in each of the series of 128 inbound 
communication cells in each master frame determines the placement 
of the inbound communication cells within the inbound delay 
compensation buffer 96. The master framing bit stream enables the 
state machine 84 to properly align master frames of 128 inbound 
communication cells in to the inbound delay compensation buffer 96 
for each of the communication links 1-N. 

Figure 6 illustrates a control message transferred over each of 
the communication links 1-N. The control message is carried via two 
master frames of inbound communication cells over each of the 
communication links 1-N. Two master frames are shown comprising 
a set of 256 communication cells shown as cell 0 - cell 256. 

The receiving communication node extracts the channel 
control bit from each communication cell in the two master frames 
and assembles the channel control bits into a control message. The 
control message comprises a channel control bit CC[0] - CC1255] from 
the two master frames of inbound communication cells. Each of the 
communication links 1-N is capable of carrying a control message 
every 2 master frames via the corresponding channel control bits. 

Figure 7 illustrates the format of a master framing bit stream for 
one embodiment. The master framing sequence is used to determine 
the relative transmission delays among the communication links 1-N 
and to realign the cell streams transmitted over the communication 
links 1-N into the inbound cell alignment buffer 86. Each master 
frame is subdivided into 16 subframes (subframe 0 - subframe 15). 
Each of the subframes 0-15 comprise the bundle framing (BF) bits from 
eight communication cells. Subframes 0-2 are shown along with 
corresponding bundle framing bits. The subframes 0-15 are delineated 
by an eight bit bundle framing pattern for each set of eight successive 
communication cells in each subframe. The bundle framing bits in 
the first four communication cells of a master frame comprise the bit 



BNSOOCIO: <WO 9608 1 20 A 1 1 > 



WO 96/08120 



PCT/US9S/10218 



- 14 - 

pattern lOlx. The next four bundle framing bits in each subfraxne 
provides a subframe number in the range 0-15. 

The bundle framing bit in communication cell number 3 in a 
master frame sequence is set to zero to indicate the beginning of a 
control channel message via the channel control (CC) bit stream. For 
one embodiment, one channel control message is transmitted every 
two master frames over each of the communication links 1-N. 

The 128 bit master framing bit stream enables delay 
compensation and realignment among the communication links 1-N 
for permissible differential delays of up to 32 milliseconds for Tl 
communication links. The permissible differential delay of up to 32 
milliseconds enables delay compensation if the communication links 
1-N are terrestrial Tl communication links. The permissible 
differential delay of up to 32 milliseconds is also sufficient if failed 
terrestrial Tl communication links are replaced with terrestrial Tl 
communication links. 

For an alternative embodiment, the transmitting 
communication node overlays another 128 bit framing pattern on top 
of the 128 bit master framing bit stream shown in Figure 7. The top 
128 bit framing pattern is overlaid onto the master framing bit stream 
shown via the don't care (x) bit positions located in the bundle 
framing bit sequence in the first four communication cells of a master 
frame which comprise the bit pattern lOlx. The don't care (x) bit 
positions of 128 sequential master framing bit streams provide the 
overlaid 128 bit framing pattern for extended differential delays. 

The 128 bit framing pattern overlaid onto the 128 bit master 
framing bit stream enables delay compensation and realignment 
among the communication links 1-N at the receiving communication 
node for permissible differential delays of up to 4 seconds for Tl 
communication links. The permissible differential delay of 4 seconds 
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enables delay compensation if a subset of the communication links 1- 
N are satellite Tl communication links or if any of the failed 
terrestrial Tl communication links are replaced with satellite Tl 
communication links. 

Figure 8 illustrates the control channel message format for one 
embodiment. The control channel (CO bit stream carries fixed length 
control messages between the master communication node 26 and the 
slave communication node 28. The master communication node 26 
assigns a link identifier (LID) to each communication link 1-N. 

The master communication node 26 transmits configuration 
information and state information to the slave communication node 
28. The configuration and state information enables the slave 
communication node 28 to reconstruct the inversed multiplexed cell 
stream transmitted over the communication links 1-N. The master 
communication node 26 also transmits control messages to the slave 
communication node 28 via the control channel bit stream to update 
configuration and state information as communication links are 
added or deleted to the group of communication links 1-N. 

The control channel message assigns one of the 
communication links 1-N as r the "primary" communication link. The 
link state (STATE) information transferred via the channel control 
message indicates whether corresponding communication link 1-N is 
blocked or deleted from the active inverse multiplex group. Channel 
control messages specify the number of active communication links 1- 
N and the sequence of link identifiers in the active inverse multiplex 
group on the communication links 1-N. Channel control messages 
identify a new primary communication link if the previous primary 
communication link fails. Control channel messages also specify 
deleted failed communication links from the active group of inverse 
multiplex communication links 1-N. In addition, channel control 
messages are used to undelete or restore previously failed 
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communication links to active group of inverse multiplex 
communication links 1-N. 

Each channel control message comprises 256 bits arranged as a 
set of 32 octets. Channel control messages carried via the 
communication links 1-N that are not primary links use only the first 
four octets out of the 32 octets in a control message. The remaining 
octets for such control messages are set to zero. 

Channel control messages on the primary communication link 
of the communication links 1-N carries the complete configuration 
information for the communication links 1-N. The octets 5-32 of a 
channel control message on the primary communication link 
indicates the logical order of link identifiers for the active inversed 
multiplexed group. The logical order of link identifiers provided by 
the channel control message enables the proper resequencing of 
communication cells from the inbound cell alignment buffer 86. 

Table 1 lists the format of a control message header for one 
embodiment. The control message header is contained in octet 1 of a 
control message. The header of a control message includes a bit that 
indicates whether the transmitting communication node is a master 
or a slave communication node and a bit indicating whether the 
corresponding communication link 1-N is assigned as a primary 
communication link or a nonprimary communication link. 
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Table 1 



BITS 


Octet 1 


HEADER 


7-4 


1111: Fixed Preamble 




1: Master side 




3 


0: Slave side 




2 


1: Link assigned as 'Primary' 
0: Link non-primary 




1-0 


00: CC msg. 

01: User to User message 





Table 2 lists the format of octet 2 of a control message which 
provides link identifier and state information. The state information 
in octet 2 indicates whether the corresponding communication link 1- 
N is deleted from an active list of communication links or is blocked 
due to a communication link failure. Octet 2 also specifies the link 
identifier for the corresponding communication link 1-N. 



Table 2 



BITS 


Octet 2 Definition Link ID & STATE 


7 


0: Link In Service 
1: Link "DELETED" 


6 


1: Link In Service 
0: Link "BLOCKED" 


5 


1: Cell Payload Scrambling disabled 
0: Cell Payload Scrambled 


4-0 


Link ID 00: Unassigned. 
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Table 3 lists the format of octet 3 of a control message for one 
embodiment. Octet 3 carries state information for the inverse 
multiplexed communication links 1-N. 

Table 3 



BITS 


Octet 3 STATE 


7 


0: Default (CC message valid) 

1: Error in CC msg. (mastership conflict, 

incorrect LIDs . . .) 


6-4 


0: Not Used 


3-0 


State Information 



Table 4 illustrates the format of octet 4 of a communication 
message for one embodiment. Octet 4 specifies the maximum number 
of active links available in the set of communication links 1-N. 



Table 4 



BITS 


Octet 4 Maxlinks 


7-5 


Not Used 


4-0 


Max. number of Links assigned 
00: Unassigned. 



Table 5 lists the format of octets 5-32 of a control message for 
one embodiment. Octets 5-32 provide link identifier and state 
information for the communication links 1-N. Each communication 
link 1-N corresponds to one of the octets in the octet range 5-32. Each 
octet 5-32 includes bits that indicate whether the corresponding 
communication link 1-N is deleted from the active group or is 
blocked. In addition, the octets 5-32 provide the sequence of link 
identifiers assigned by the master communication node 26. 
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Table 5 



BITS 


Octet 5 ... 32 Link ID & STATE 


7 


0: Link In Service 
1: Link "DELETED" 


6 


0: Link In Service 
1: Link "BLOCKED" 


5 


0: Cell Payload Scrambling disabled 
1: Cell Payload Scrambled 


4-0 


LID (Link ID) assigned by Master at connection time 
00: Unassigned. 



Figure 9 illustrates a complete master framing sequence 
provided by the bundle framing bits and a complete control message 
provided by the channel control bits of each communication cell 
transferred over each communication link 1-N. A master framing 
sequence comprises 128 communication cells on each communication 
link 1-N. A control message comprises 2 master framing sequences 
for a total control message length of 256 bits. 

Figure 10 illustrates the architecture of the inbound delay 
compensation buffer 96 for one embodiment. The inbound delay 
compensation buffer 96 comprises a set of link buffers 1-N. The link 
buffers 1-N correspond to the communication links 1-N. For one 
embodiment, each of the link buffers 1-N comprise a circular buffer. 

The state machine 84 sequences each communication cell 
received over each communication link 1-N into the corresponding 
link buffer 1-N according to the corresponding master framing 
sequence. For example, the state machine 84 sequences each 
communication cell received over the communication link 1 into the 
link buffer 1 according to the master framing sequence for the 
communication link 1. The state machine 84 determines the location 
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in the link buffer 1 for each inbound communication cell on the 
communication link 1 by examining the value of the corresponding 
bundle frame bit for the inbound communication cell in relation to 
the overall master framing sequence* Inbound communication cells 
on the communication links 2-N are sequenced in a similar manner 
into link buffers 2-N respectively. 

The state machine 84 maintains a set of link write pointers 
(LWP) 1-N. Each link write pointer 1-N provides a write pointer into 
the corresponding link buffer 1-N. The state machine 84 updates the 
link write pointers 1-N as inbound communication cells are 
sequenced into the link buffers 1-N. The state machine 84 filters out 
idle inbound communication cells over the communication links 1- 
N. 

Inbound communication cells are sequenced out of the 
inbound delay compensation buffer 96 and stored in the inbound cell 
buffer 86 in the original cell sequence using a cell read pointer. The 
cell read pointer follows the link write pointers 1-N around the link 
buffers 1-N in a circular fashion. The cell read pointer follows the link 
write pointer 1-N that corresponds to the communication link 1-N 
having the longest delay in communication cell transmission between 
the communication nodes 26 and 28. The master framing sequence 
on each communication link 1-N provides an indication of the 
relative phase delay for each of the communication links 1-N. The 
cell read pointer is only updated after an inbound communication cell 
corresponding to the communication link 1-N having the longest 
delay is stored into the inbound delay compensation buffer 96 in the 
appropriate link buffer 1-N. 

Figures 11a and lib illustrate an example arrangement of 
inverse multiplexed communication lines and corresponding control 
messages. For the example shown, the communication links 1-N 
correspond to communication links 1-5. The link identifiers assigned 
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to the communication links 1-5 are link identifiers 7-11. The active 
communication links correspond to link identifiers 7, 8, 10, and 11. 

Figure 11a illustrates a sequence of communication cell 
numbers from the outbound cell buffer 60. The cell number sequence 
comprises cell numbers 11, 10, 9, 8, 7, 6, and 5. The outbound 
communication cell sequence on the communication links 1-5 is 
assigned in a round robin fashion over the active communication 
links corresponding to the link identifiers 7, 8, 10 and 11. 

Figure lib illustrates the format of a control message carried 
over each of the communication links corresponding to the link 
identifiers 7,8, 10 and 11. For this example, the communication link 
corresponding to the link identifier equal to 7 is designated at the 
primary communication link which carries the entire configuration 
message. 

In the foregoing specification the invention has been described 
with reference to specific exemplary embodiments thereof. It will, 
however, be evident that various modifications and changes may be 
made thereto without departing from the broader spirit and scope of 
the invention as set forth in the appended claims. The specification 
and drawings are accordingly to be regarded as illustrative rather than 
a restrictive sense. 
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Claims 

What is claimed is: 

1- An asynchronous transfer mode communication method, 
comprising the steps of: 

transferring a series of communication cells over each of a set of 
communication links while manipulating a framing bit in each 
communication cell such that the framing bits provide a 
predetermined framing bit stream for each communication link; 

receiving the communication cells over the communication 
links and aligning the received communication cells from each 
communication link according to the corresponding framing bit 
stream. 

2. The method of claim 1, wherein the step of aligning the 
received communication cells includes the steps of: 

determining a differential delay among an active set of the 
communication links using the framing bit streams; 

if the differential delay is greater than a predetermined delay 
limit, then deleting the communication links that exceed the 
predetermined delay limit from the active set of the communication 
links. 

3. The method of claim 1, further comprising the step of 
manipulating a control channel bit in each communication cell while 
transferring the communication cells over each communication link 
such that the control channel bits provide a control message for each 
communication link. 

4. The method of claim 3, wherein the control message specifies a 
logical identifier for each communication link. 

5. The method of claim 3, wherein the control message specifies 
an ordered list of the logical identifiers such that the ordered list 
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indicates a multiplexed sequence of transfer of the communication 
cells over the communication links. 

6. The method of claim 5, further comprising the step of deleting a 
failed communication link from the set of communication links by 
deleting the logical identifier for the failed communication link from 
the ordered list. 

7. The method of claim 6, further comprising the step of restoring 
the failed communication link to the set of communication links by 
adding the logical identifier for the failed communication link to the 
ordered list. 

8. An asynchronous transfer mode communication system, 
comprising: 

means for transferring a series of communication cells over 
each of a set of communication links while manipulating a framing 
bit in each communication cell such that the framing bits provide a 
predetermined framing bit stream for each communication link; 

means for receiving the communication cells over the 
communication links and aligning the received communication cells 
from each communication link according to the corresponding 
framing bit stream. 

9. The communication system of claim 8, wherein the means for 
aligning the received communication cells comprises: 

means for determining a differential delay among an active set 
of the communication links using the framing bit streams; 

means for deleting the communication links that exceed a 
predetermined delay limit from the active set of the communication 
links if the differential delay is greater than the predetermined delay 
limit. 
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10. The communication system of claim 8, further comprising 
means for manipulating a control channel bit in each communication 
cell while transferring the communication cells over each 
communication link such that the control channel bits provide a 
control message for each communication link. 

11. The communication system of claim 10, wherein the control 
message specifies a logical identifier for each communication link. 

12. The communication system of claim 10, wherein the control 
message specifies an ordered list of the logical identifiers such that the 
ordered list indicates a multiplexed sequence of transfer of the 
communication cells over the communication links. 

13. The communication system of claim 12, further comprising 
means for deleting a failed communication link from the set of 
communication links by deleting the logical identifier for the failed 
communication link from the ordered list. 

14. The communication system of claim 13, further comprising 
means for restoring the failed communication link to the set of 
communication links by adding the logical identifier for the failed 
communication link to the ordered list. 

15. A communication system, comprising: 

first communication node coupled for communication over 
each of a set of communication links according to an asynchronous 
transfer mode protocol, the first communication node accessing a 
series of communication cells according to an outbound cell sequence 
and transferring the communication cells over the communication 
links according to an inverse multiplexed sequence; 

second communication node coupled to receive the 
communication cells over the communication links and reconstruct 
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the outbound cell sequence of the communication cells while 
compensating for differential delays among the communication links. 

16. The communication system of claim 15, wherein the second 
communication node determines a differential delay among an active 
set of the communication links using the framing bit streams and 
deletes the communication links that exceed a predetermined delay 
limit from the active set of the communication links. 

17. The communication system of claim 15, wherein the first 
communication node transfers the series of communication cells over 
the communication links in the inverse multiplexed sequence while 
manipulating a framing bit in each communication cell such that the 
framing bits provide a predetermined framing bit stream for each 
communication link. 

18. The communication system of claim 17, wherein the second 
communication node receives the communication cells over the 
communication links and aligns the received communication cells 
from each communication link according to the corresponding 
framing bit stream. 

19. The communication system of claim 18, wherein the first 
communication node manipulates a control channel bit in each 
communication cell while transferring the communication cells over 
each communication link such that the control channel bits provide a 
control message to the second communication node. 

20. The communication system of claim 19, wherein the control 
message specifies a logical identifier for each communication link. 

21. The communication system of claim 19, wherein the control 
message specifies an ordered list of the logical identifiers such that the 
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ordered list indicates a multiplexed sequence of transfer of the 
communication cells over the communication links. 

22. The communication system of claim 21, wherein the first 
communication node deletes a failed communication link from the 
set of communication links by deleting the logical identifier for the 
failed communication link from the ordered list. 

23. The communication system of claim 22, wherein the first 
communication node restores the failed communication link to the 
set of communication links by adding the logical identifier for the 
failed communication link to the ordered list. 
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